Structural Symmetries from Motion for Scene Reconstruction and Understanding

The identification and description of partial symmetries in man-made structures is a powerful tool to improve the quality of 3D reconstruction from unordered images and to enable high-level understanding of scene geometry. In this work we propose an approach to identify symmetries and exploit them in Structure from Motion (SfM). Our first contribution is a symmetry detection approach that uses the 3D geometry of the scene as well as 2D appearance clues. We show that a particular parametrization of the transformation space (space in which each point represents a candidate symmetry relation) exposes the dominant symmetries in the scene. Then, we use appearance information to prune incorrect symmetry hypotheses. The second contribution is a constrained bundle adjustment (CBA) scheme that jointly optimizes for the best 3D reconstruction and the symmetry generators. Contrarily to related work on CBA, our approach models n-fold (rotational and translational) repetitions of architectural elements, and allows estimating a generative model of the 3D geometry. Experimental results confirm that our method can correctly identify and exploit partial symmetries in noisy and sparse SfM datasets.